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ABSTRACT 

During the years since 1982 when the Job Training 
Partnership Act (JTPA) was passed, program and management practices 
have evolved in ways quite different from JTPA's much^criticized 
predecessor, the Comprehensive Employment and Training Act (CETA) . 
The most significant evolution has been the strongly results-driven 
nature of the program accompanied by the development of formal 
national, state, and locally administered systems of outcome measures 
and standards for aggregated program achievement performance 
standards. Two other practices that have evolved ovtr the several 
years of full implementation of JTPA have been performances-based 
contracting and competency-based programming. The performance 
standard approach has much in its favor, notwithstanding important 
issues about it£ effects on the design of programs and the selection 
of clients. The performance contracting system, may conceal rather 
than reduce administrative costs, and it may result in serving only 
those clients easiest to train and place; however, it has merit. 
Finally, the competency-based program movement has been slower to 
develop and is harder to evaluate, although it has good points. These 
three aspects of the JTPA have implications for adult vocational 
education. JTPA has shown that is possible to establish a program 
driven by clear, measurable outcomes, and vocational education should 
move m that direction. At the same time, JTPA could use the training 
expertise of vocf-tional education. (46 references.) (KC) 
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When the Job Training Partnership Act was passed with some fanfare in 1982, it 
culminated a bi-partisan policy salvage job of significant proportions. The new federal 
Administration had Utde interest in employment and training for the disadvantaged, and had only 
to cite the weU-publicized difficulties of CETA as the basis of its skepticism. CETA had peaked 
in annual budget at over $12 billion in 1979, and had become— for reasons principaUy of bad 

management and unlucky public relations almost from its beginning in 1974 a favorite 

whipping boy of critics of "social programs.'* 

The new Administration, looking for opportunities to cut programs from the budget, first 
drastically cut CETA's budget, then opposed its renewal when it expired in 1981. It was not 
surprising to sec first, opposition, then— in the face of clear support in both parties, in both 
branches, for doing something in employment and training— only lukewarm support for new job 
training legislation to replace CETA. Nonetheless, by mid 1982, Administration support came in 
part because a bipartisan consensus was clearly building for a formulation acceptable both to 
House and Senate sponsors and to the few in the Administration who favored employment and 
training legislation. 

The elements of this formulation— reflected in the Bill which was finally passed and signed 
in October, 1982 — included elimination of public service employment and most forms of 
subsidized work experience, the elevation of the role of the private sector through Private 
Industry Councils, increased authority for the states, and a variety of measures designed to 
promote more rigorous, less expensively administered, results-oriented management 

Coupling these elements together with a reduced budget request (less than $4 billion) maoe 
it possible to sell JTPA as a lean, mean, training-oriented, anti-make-work, pro-business, 
bottom-line driven initiative in the spirit of public-private parmership. This was not "son of 
CETA," but a more distant relative. And so it has been in implementation. 

Despite normal inertia at all levels, from the U.S. Department of Labor through local 
Service Delivery Areas, JTPA — now in its seventh year, but only in its fifth year of full-scale 
implementation — has developed into quite a different employment and training system from its 
predecessor. States have become major players, often at the expense of local governments, 
which were firmly m charge during CETA. The federal Department of Labor spent the first 
several years of JTPA's existence avoiding any programmatic role at all, short of toting up 
results, and most states were perfectly willing to pick up the slack. While the organized private 
sector is hardly in the driver's :>cat envisaged by some, certainly many more private employers 
are involved in JTPA than were during CETA. Aside from the drastic reduction in budget, which 
has had the most dramatic impact, the instinitional and political changes which have foUowed are, 
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in some ways, the more noticeable. Quieter, less noticed still, but in some ways even more 
significant, arc the changes in program and management practices which have resulted from 
JTPA's provisions calling for more outcome-oriented accountability — notably the requirement 
that governors monitor the performance of SDAs against certain outcome standards. CETA was 
heavily procedural, that is focused on participatory requirements for "significant population 
segments," :eting devices, and slot levels—and almost before it got started it was burdened 
additionally with such confusing priorities as "public service employment for countercyclical 
economic stimulation," an administrative and public relations disaster. 

JTPA, on the other hand, has been almost fanatically outcome-driven, and both 
Administration and Congress have, remembering CETA, strongly resisted "confusing" 
amendments even when technical and even programmatic adjustments were badly needed. 

The most significant outcome-oriented practice has been the development of formal 
national, state, and locally-administered systems of outcome measures, and standards for 
aggregated program achievement This so-called ^^performance standard" system-— driven by a 
nationally-derived set of outcome numbers against which the performance of local administrative 
entities, called Service Delivery Areas, or SDAs, (and, in turn, states) are measured— has 
become the basis for judgement about the effectiveness of local programs, and, when they are 
aggregated, about the system as a whole. This rigorously-applied set of measures and standards 
has had the effect of turning the system on its head to comply. The measures have recendy been 
expanded, and there have been annual adjustments in die standards, but in general there has been 
great consistency over the several years of JTPA, providing a good basis of experience with 
outcome measures for a national program. 

Two other practices that have evolved over the several years of full implementation of 
JTPA have cither hardened or mitigated the impacts of the performance standard system. The 
first is a set of approaches to "performance-based contracting" which, while not mandated— or 
even mentioned — by JTPA, has quite literally swept the system in the last three years. The 
second practice, authorized by JTPA, but littie emphasized by the performance standard system 
in the first few years, is the development of competency measures for program outcomes. To 
some extent, the movement in JTPA towards competency measures, and in turn to competency- 
oriented training, though specifically authorized only for youth, has been a reaction against the 
performance standard system, which has emphasized job placement over other outcomes. 

These three practices — performance standards, performance contracting, and competency- 
based programming — all have implications for the management of the vocational education 
system, and thus will be the focus of this brief review. I will argue that the performance standard 
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approach has much in its favor, notwithstanding important issues about its effects on the design 
of programs and the selection of clients. With appropriate modifications for difference of 
purpose, it ought to be emulated in large part by revised vocational education legislation. 

I will describe the performance contracting system which has evolved in the last several 
years, and examine the reasons for its development, but will suggest that in local application the 
approach may have accumulated some unintended but important negatives. It may simply 
conceal, radier than reduce, actual administrative costs, certainly not a result intended by the 
designers of JTPA. As performance contracting has developed as a local response to meeting 
performance standards, it appears that the procedure may in some instances have had the effect of 
screening out harder-to-serve, more disadvantaged (and therefore often more expensive) 
applicants by encouraging or permitting local contractors to seek quicker, cheaper results, with 
likely compromises in quality of services. Yet its simplicity and apparent efficiency argue for 
fmding other ways to compensate for these unintended effects without eliminating performance 
contracting. 

The competency-based programming movement is a slower and harder development — 
because it has to do with actual design of cunicula and services — which has developed more in 
the youth area (where, under the nabric of "employability enhancement," it is for the first time 
this year an authorized national performance measure). Though most localities have used the 
approach in some programs, the concept has not spread as quickly through the system as 
performance contracting has, largely because it is hard to do. But recent changes in the 
performance standards system may have the effect of encouraging more extensive development, 
giving local administrators the wriggling room to succeed according to performance standards yet 
still serve more complicated and exper^ive clients. 

Taken in appropriate combination, diese three outcome-oriented management and program 
planning devices may indeed evolve into a results-oriented system of training and education for 
the disadvantaged. I will suggest that they have not yet matured but that they are worth emulating 
by vocational education, given certain cautions. 

I will discuss these three aspects of a performance system under JTPA one at a time, 
indicating what is presently understood about each, presenting data and opinions diawn from a 
survey of nearly 100 SDAs and twelve states as well as from other, earlier reviews of JTPA, and 
from a series of free-ranging interviews with experienced local and state practitioners. Finally, I 
will make a few recommendations for vocational education to be drawn from this experience. 

At this point, a final introductory caveat: this is a personal review, written by one who has 
worked with CETA, worked on planning for JTPA, and who has watched JTPA's 
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implementation closely. The surveys and interviews were conducted to ascertain the extent of use 
of certain practices in the field, and to solicit the opinions of local and state officials about their 
evolution and their meaning. I posit no new models, and pretend no great science. To the 
contrary, the survey was open-ended, the questions searched for opinions more than numbers, 
and the analysis of the results was only crudely statistical. But we did talk to more than a 
hviidred people who work with JTPA at the local level, another twenty or so who work at the 
state level, and a dozen or more who have watched or worked with JTPA as a national program. 
However, both the conclusions drawn from these conversations and the recommendations for 
what they may mean for adult vocational education are my own. 

Performance standards and the simplicity impulse 

Section 106 of the Job Training Partnership Act consists of two pages of requirements 
introduced by die following phrase: "Congress recognizes tiiat job training is an investment in 
human capital and not an expense. In order to determine whether that investment has been 
productive . . The Act then specified that the efficiency of the investment would be measured 
by "increased employment and earnings and by reduced welfare dependency.*' This, contrasted 
by all to CETA, represented a shift from counting inputs to measuring outcomes, from 
influencing program through defining procedural requirements to doing so by measuring 
outcomes. The E)epartment of Labor was charged with designing measures and establishing 
standards by which states, and in turn local programs could be evaluated and judged. 

As apparentiy intended by Congress and practiced by the Department of Labor (and 
overseen vigorously by the Office of Management and Budget), these measures and standards 
were to be few, were to be easy to aggregate and explain, and were to be easily used as a device 
to communicate effectiveness of the program to a cynical world. Therefore, though no one ever 
said it out loud, they had to remain simple. The shadow of CETA loomed large: it was too 
complicated, too procedural, too social-prog.-am-y, ultimately unmeasurable, and not susceptible 
to the bottom-line mentality presumed to be held by private business people. Though the 
measures differed for youth (14-21) and adults (22 and up), the first set issued by DOL was 
straightforward: 

Adult measures 

entered employment rate 
cost per entcroi employment 
average wage at placement 
welfare entered employment rate 



Youth measures 

entered employment rate 
positive termination rate 
cost per positive tennination 

Federal standards were esubUshed, suggesting numerical performance levels, and an adjustment 
methodology was developed, permitting governors to modify the standards by application of 
local measures which might suggest either greater difficulty of achievement or other priorities. 
Though the actual standards have changed over the years, they have remained fairly consistent: 
the entered employment rate standard has been around 60% for adults, 40% for youth; the 
welfare entered employment rate has been around 50%, average cost per adult entered 
employment a little over $4,000, and average wage a little less than $5.00/hr. The added youth 
standards have required that roughly 75% of all participants "terminate positively," that is either 
enter employment, or demonstrate certain approved competency levels, and that those standards 
be achieved at an average cost per positive termination of around $5,000. The standards have 
been set so that about 75% of all SDAs achieve them, and adjusted upward or downward for 
each new year depending upon the previous year's experience. 

In reviewing programs and establishing standards, governors were to provide a statement 
of goals and objectives for the state, numerical standards against which to judge local and 
statewide performance, and could, if they wish, propose an "adjustment methodology" which 
would allow Service Delivery Areas (SDAs) to adjust standards to reflect local conditions. In 
addition, they could propose possible additional measures for statewide priorities and an 
incentive and sanctions policy for good or poor local performance against the standards. 

In turn, the local Service Delivery Areas (there are some 600, oriented to labor market 
areas, covering the country), would: 

• prioritize local needs 

• propose local adjustments to performance standards 

• decide who administers the program locally 

• select service providers, and 

• design and cany out service strategics in each area 

And so it began, fint applied to the 'Transition Year," 1983-84. (Until then, CETA 
continued to operate under Continuing Resolution.) States worked hard to adapt to a newly 
prominent role, the federal Department of Labor tried to back off its former aggressive procedural 
oversight, and locals tried to toe the line. In truth, according to our interviews of senior state and 
local people who were working in the system at the time, while there were concerns from the 
beginning about the possible effects of such placement-oriented performance standards, criticism 
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was muted. As one put it, "most of us (employment and training professionals) were so relieved 
that there was legislation at all that we tended to keep our mouths shut about our reservations and 
try to make sure that we met or exceeded any standard the feds issued" (Brandeis interview, 
1988). 

And meet them they did. From the first year, the system has me! or exceeded, on the 
whole, the numerical standards issued by the U.S. Department of Labor. DOL consciously set 
out to issue "minimally-acceptable'* numerical levels so that the standards would not unduly 
distort program design. Whiic the standards have varied slightly from year to year, and generally 
have trended upward, they have remained in the same neighborhood for the first five years. 

Some states and SDAs have modified standards according to local conditions, but not 
many, and not by much. Of the SDAs wc interviewed, 22% reported that they had poposed 
modified standards at least once in the five preceding years. On average the request was for only 
a "point or two"* in adult entered employment rates, never for cost standards, and in two cases 
there had been requests for a reduced level of expenditures on youth (not a performance 
standard, but rather a JTPA requirement with waiver procedures). Only two of the twelve states 
we interviewed ever modified the feSfcral standards statewide. Massachusetts raised the numerical 
standards "across the board," and South Carolina reported using federal standards as a 
"minimum, with a tiered incentive system rewarding SDAs for performance exceeding the 
standards" (Brandeis interviews, 1988). The remainder simply applied the standards issued by 
DOL. 

Thus, even though elaborate provisions had been made for "the Governor" to propose 
modifications, and suggestions were made for regression formulas to be used to factor in local 
and statewide circumstances, it appears that the course of least resistance was followed in most 
cases by most states; and sub-state modifications, while relatively more common, were modest. 
Accordingly, it is possible for the purpose of this review to make generalizations which can be 
presumed to apply to most SDAs and in most states. 

It is clear from our 1988 survey, as well as from most other sources, that the performance 
standards system so far has woiked. A few bottom line measures were proposed, numerical 
standards followed, nearly every state and most SDAs adopted them as proposed, and most 
organizations involved in JTPA achieved the proposed standards for program outcomes. While 
the standards were keyed fairly low (at the 2511-1 percentile), to make compliance likely, the 
attitude of local and state administrators was that whatever they were, they had to be met. It was 
a matter of political survival for the system. As one SDA suff director told a 1985 study of 
JTPA: 
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"Our fint job was to prove that JTPA wasn't CETA. That means to be 
efTicient and avoid fraud and abuse. We've had no fraud and abuse. And 
our placement rates and costs beat the federal standards — that proves 
we're efficient" (Ciinker-Walker, 1985) 

Policy lessons about performance measures and standards 

There is an underlying policy question about the application of performance standards 
under JTPA. That is, do performance standards achieve the "investment in human capital" 
agenda of the Act, or docs the approach artificially force sutes and localities to opt for quick 
numbers which will yield only short-term results? Unfortunately there are no reliable data which 
will answer that most basic question. There are pretty good data on which cUents were served, 
on where how many of them were placed and at what salary, and on how much the sendees they 
were provided cost But there is little systematic information on what services were actually 
provided and virtually none at all on who else might have been served had the programs been 
designed or supported differendy— or even on who applied for services but was rejected on 
grounds that they might not succeed. Instead, we must rely on very partial official daU on a few 
aggregate measures, and on opinion and self-reporting by professionals involved in the program. 

When we began to ask for those opinions, we started with a few questions in mind. Were 
the performance measures being applied the right ones? Were the standards high enough? Did the 
standards achieve the desired management effects? Did they affect the design of services? The 
choice of participants? Were there important negative side effects? We posed those questions in 
various ways to the SDA and state people with whom we spoke; their answers, our own 
experience and a close reading of other reports helped to form the replies given below. 

Are the measures the right ones? 

The jTPA system has been truly converted to performance toward outcome measures, and 
in a remarkably short time. While the: e are a few holdouts for a more elaborate set of outcome 
measures — several measures for youth competency gains not now used, for example — almost 
everyone agrees that entered employment rates, welfare reduction rates, employment wages and 
cost measures make sense. Fully four-fifths (82%) of those wc interviewed thought that the new 
national measures and standards "moved in the right direction" — towards long-term 
employability — while a handful echoed one opinion that they "watered down a tough system that 
works" (Brandeis survey, 1988). 
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Aire the standards high enough? 

There is near consensus that linking each year's performance standard with last year's 
actual perfonnance makes sense, and that the standards which result are "about right, because we 
are pushed each year to do a litUe better than last year. . . (Brandeis survey, 1988). There is 
some objection to the wage standard, reflecting the concern that it is too easy to settle for poor, 
low-paying jobs — "$5.00 an hour just isn't a veiy good job, and with that as a required average, 
it means we are setding for some jobs at even less, and I can't see where that's much progress 
for most of our folks-— even die welfare moms can do better than that staying on welfare. . . 
(Ibid.) While it is not scientific, because the question was not pursued across the board, it 
appears that the SDAs differ on this question, with those in strong state or local economies 
feeling the wage standard may be too low, while those in weaker local economies think it is 
about right, or even high. The most conmionly expressed concern about the standards is about 
the unintended effects on the selection of clients, which I will discuss below. 

Have the standards achieved the desired management effects? 

Again the answer is largely positive. In view of the reputation of CETA, mentioned by 
almost everyone with whom we spoke, it was essential to be able to claim some 
accomplishments. "Meeting our performance standards has helped us with our local employers, 
especially our PIC members, and turned around our image with the general public. I'd never 
want to do it another way," said one SDA director, "but I'm still privately worried that it has 
changed the way we target our clients — getting folks we are surer to succeed with rather than the 
ones who need our help the most" (Ibid.) 

Has the system affeaed the design of program services? 

The principal effect of the performance standard system appears to have been the 
development of performance contracting, which is important enough to warrant a separate 
discussion later in this review. Most SDAs have moved to subcontract nearly all program 
services, and most have established perfonnance contracting procedures, distancing themselves 
from actual program design thereby. Most appear to be generally comfortable with the oversight, 
management oriented role subcontracting assigns them, but several expressed concern about the 
effects. One large-city SDA director described his p)erfcrmance contracts as "a black hole. As 
long as they report achievement of their contracted numbers, and my monitors confirm that the 
people and their jobs are for real, I know hardly anything about the content of the services we 
offer. It makes me very nervous," (Ibid.) 
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Have performance standards ejected the choices about who participates in JTPA ? 

This is the area of greatest controversy in JTPA. Do business participation in JTPA, the 
impulse for efficiency and cost saving, the performance standard system, and performance 
contracting conspire to impel the system to serve the easiest among the eligible population to 
serve? In simpler form, the question even spent a day or two in national presidential politics, 
associating the question with Senator Quayle, one of JTPA's two Senate sponsors and widely 
regarxled as its principal sponsor. 

Disaggregated, the question generally locuscs on the contribution of single factors— the 
role of private industry council members, of performance standards, of performance contracts, of 
budget reductions— to what appears to be a subtie but generally accepted practice of "serving 
those most likely to benefit from our services.** The well-regarded Grinker- Walker review of 
early implementation concluded that "Under JTPA there has been an emphasis on quick results 
and inexpensive placements" and attributes the impulse to private sector influence, and the 
tendency to seek compliance with requirements more than to meet long-term goals, with "an 
impact on their choice of programs and applicants." (Grinker- Walker, 1985) 

Three years later, perhaps reflecting shifts in the way the standards are administered 
nationally, state and SDA officials are not so sure. Programs are modcsUy longer in duration 
(about 18 weeks on average, up from the 14.2 documented by Grinker- Walker), and the majority 
of people to whom we talked either thought that "crcanung** was not taking place in their 
jurisdiction ("over 75% of our participants are welfare recipients; how can that be creaming?"; or 
that "it probably used to occur, but not this year— we'll be targeting higher-risk adults, more 
welfare recipients, and more school dropouts" (Brandeis survey, 1988). 

And indeed that appeais to be the trend In Program Year 1988, for the first time, DOL has 
issued new program measures and new standards, creating a "menu" of twelve standards from 
which states and SDAs may choose eight. Identifying the new issuance with the goal of 
"increasing services to at-risk individuals, particularly youth, to encourage provision of training 
which is relevant to long-range employ ability, and to encourage increased provision of basic 
skills training," the Department also intioduccd new measures for post-program achievement, 
including job retention. 

It seems likely that the impulse for simplicity which led to concentration on a few basic 
outcome measures, and the essentially short-term character of those measures, more than the 
existence of measures and standards themselves, did contribute to a "creaming" trend in 
participant selection in JTPA. Bui it was hardly universal across states and SDAs, and it appears 
that certain of the standards made a bigger contnbution than others. For example, the National 
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Commission on Employment Policy concluded in a review published just this Fall that states and 
localities which focused on exceeding (by coming in under) the cost standard "tended to serve 
fewer hard-to-serve cUents ... and tended to design less intensive services." (NCEP, 1988) 

Similarly. SDAs surveyed for the NCEP study which emphasized exceeding, rather than 
simply meeting, the placement and other standards also tended to select easier people to serve and 
to offer fewer basic skills training and other long-tenn-oriented services. And more of both took 
place in states where incentives emphasized (and rewarded) exceeding the standards. On the 
other hand, where states offered incentives for targeting service to particular populations (e.g., 
school dropouts or welfare recipients), SDAs met those objectives with longer-term, more 
expensive programs. 

All of which seems to make two points. First, it is not at all clear that the performance 
standards system itself produces the "creaming" effect which has so concerned critical observers 
of JTPA, even though in certain applications it clearly contributes to that effect See*, d. k 
appears that by emphasizing diftcrent objectives (including **anti-creaming" measures like 
population targeting) through incentive payments, states and localities can apply performance 
standards to any measurable objective. 

Should Vocational Education pay attention to this experience? Clearly, yes. There are 
important differences in the situation faced by re-designers of vocational education. The federal 
dollars are a less significant portion of the whole program, to be sure, and the relationship 
between the Department of Education and the state departments of education whi ;h will 
administer federal funds certainly differs from DCL's more commanding relationship with states, 
if only because of fhe dollar difference and tJic historic independence of state departments of 
education. But notwithstanding those differences, performance standards tied to performance 
objectives, with incentives both at the state and local level tied to selected priorities, probably 
makes sense for most of the same reasons they have been so successful under JTPA. A "bottom- 
line" mentality makes for better public relations, more serious business involvement, and more 
programmatic attention to outcomes than either to inputs or procedures. While JTPA's experience 
has been mixed on the subject of targeting the most disadvantaged clients, it has made clear that 
states and the federal government can "tweak" the system by providir g for adjustment 
procedures and incentives for meeting desired outcomes. That has appeal for both managerial and 
public policy reasons and ought to be incorporated in any revision of national vocational 
education policy and into practice at the state and local levels. 
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Performance Contracting: ruse or tool? 

If perfonnaiice measures and standards have been the federal government's way of seeking 
accountability for outcomes, and adjustment methodologies and incentives have been the states* 
method for incorporating their own priorities, peffamance contracting has been the local service 
delivery system's response. 

Even though there is neither mention nor implied permission in the JTPA legislation for 
such an approach, fully four-fifths (81%) of the SDAs we surveyed reported using performance 
contracts with their subcontractors, defined simply as a contracting procedure in which( typically) 
:in SDA ties all or part of its payments to subcontractors to the achievement of agreed upon 
measurable outcomes. Since fewer than 15% of SDAs provide all of their own training, "in 
house," as it were, and two-thirds subcontract at least half of all their training funds, this is a 
substantial penetration of the system. 

Unlike the data on the system's response to performance standards which, though available 
only on a few simple measures, are extensive, official, and readily accessible, not much was 
systematically available to us on performance contracdng. 

We began with some experience with state and local practitioners and had formed 
beginning impressions, but decided to do a survey of a fair sample of SDAs to find out more 
about both this practice and about competency-based programming. 

Dimng July ard August of 1988, we surveyed 84 Service Delivery Areas and twelve states 
by telephone and reviewed the results of several related surveys undertaken by other researchers. 
A closed ended instrument was utilized for our survey, and a purposive sample drawn in order to 
obtain a wide range of SDA types and labor markets. Since random sampling was not utilized, 
readers should not assume that the proportions of responses presented in this paper are 
scientifically representative of the entire group of over 600 SDAs. However, since the 
conclusions seem to square with the findings of several other surveys over the past few years, 
and with anecdotal evidence provided by an active network, we feel comfortable with the results. 

We we:e interested in several aspects of performanct contracting: the extent to which the 
practice was used in local service delivery areas, why local officials who used it did so, what 
they perceived as its benefits from both their own and the clients' points of view, and what they 
perceived as the disadvantages, from both points of view. 
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How widely is performance contracting used? 



As shown below, both our survey and other research indicate that the vast majority of 
JTPA SDAs now use performance contracting for at least some of their contracts with service 
deliverers; our survey suggests that this mechanism is used most frequently for vocational skills 
training in the classroom setting, and least often for job search assistanix. The combined results 
from several questiois yield the following summary: 

Exhibit i 

Extent of use and setting for performance contracting 
Brandeis Survey (PY 1988)* 



One or more subcontracts 80. 9 % 

Basic educadoi 50.0% 

Pre-cmploy mcnt/work maturity 53.1% 

Oassroom vocational skills 73.4% 

On-the-Job 48.4% 

Job search assistance 32.8% 
* N= 68 respondents 



In addition to our own survey. Bailis (1985) conducted a smaller survey with a stratified random 
sample of 28 SDAs to discover in what situations such contracts are used. As early as three years 
ago, over 90% reported using performance-based contracts for class-sized vocational skills 
training contracts (the type most likely to be applicable to adult vocational education), while a 
smai er but still significant number ( 80.8%) reported using them for other types of subcontracts. 

Why c id administrators opt for performance contracting? What do they consider the advantages 
to be The benefits? 

SDA officials who have chosen performance contracting tend to say that they have done so 
in order to make it easier to achieve their performance standards or to simplify program 
administration; they do not generally say they have adopted this mechanism to improve the 
quality of training. 
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As shown below (Exhibit 2), the most frequently -cited advantages of performance 
contracting for adiiiinistraton were three: it allowed them to reduce apparent administrative costs 
by including the costs of program service administration in fixed unit prices for subcontracts, it 
aUowed for ''greater accountability," and it aUowed them to key subcontract performance and 
actual payments to accomplishment :f performance outcomes. 

Exhibit 2 

Advantages of performance contracting 
(from administrators* point of view)* 



Reductions in (or increase in ability to 59.6% 
hide) administrative costs 

Better accountability 57.7% 

Easier to meet performance standards 48. 1 % 

Easier administration 36.5% 

Better training 21.2% 

Other 19.2% 
*N = 68 



As indicated below (Exhibit 3), fewer than half of the administrators adopting performance 
contracting thought that iti benefits included either better training or better placement. Here is 
how the responses summarize: 

Exhibit 3 

Benefits of performance contracting 
from administrators' point of view)* 



Better training 40.4% 

Better placement in jobs 36.5 % 

Better recruitment of clients 5.8% 

Other benefits 7.7% 
*N = 68 



One SDA director summarized the views of her colleagues best when she said, "Look, the 
way I see it, perfonnance contracting isn't about services, it's about management. It costs more 
than the law allows formally to run these programs, so we include legitimate costs of 
administration in fixed price contracts and call it all training costs. And we have to meet 
performance standards, so the contracts are drawn so that we pay for performance according to 
the standards we have to meet It's as simple as that" (Brandeis interviews, 1988). 
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What were the disadvaruages and problems with the approach? 



When viewing it from their own point of view, SDA staff tend to see three interrelated 
problems with pcrfonnance contracting: (a) producing excessive concern for numbers, "churning 
people out of the system in order to meet pciformancc standards" while (b) providing little 
incentive for "quality,'* but, on the contrary, (c) providing disincentives to serve the hardest-to- 
place. (Exhibit 4) 

As is shown in Exhibit 5, when pressed to review these disadvantages from the clients' 
vantage point* the administrators stressed (a) "creaming" — the likelihood that certain applicants 
were being excluded from programs because of performance contracting, and (b) effects of the 
preoccupation with numbers on the quality of training. 

Exhibit 4 

Disadvantages of performance contracting 
(from administrators' point of view)* 



Creaming 27.9% 

Excessive concern with numbers 27.9% 

Problems with timing or level of 

reimbureements 25.0% 

Lack of standardization 8.8% 

Difficulties in monitoring 8.8% 

Other Drawbacks 3? 3% 

*N = 68 



Exhibit 5 

Administrators* judgments about disadvantages 

from clients' point of view* 

Creaming 32.3% 

Concerns about quality or quantity 

of training received by clients 30.8% 

Concerns about churning of clients 20. 6% 

Lack of incentives to provide 

support services 8.8% 

Others 8.8% 

N = 68 
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WhiJe roughly two-thirds (69%) of the SDA repitscntatives said that they had systems in 
place to "assure quality** and "avoid creaming" in perfomance contracts, there are still two 
causes for concern. First, this means that about one third of those we interviewed simply put out 
the contracts and relied on invoicing systems x> track accomplishments. They reported no system 
to guarantee quality. Secondly, many of die SDAs that did have systems in place were relying 
only on written documentation or contract provisions to assure quality, and reported no physical 
or file monitoring or audits. 

Finally, both SDA people and state officials we interviewed suggested that many JTPA 
subcontractors have found peiformancc-based contracting problematic. More than half (52.9%) 
of the SDA representatives in our sample indicated that some or all of their subcontractors felt 
this way, primarily because of funding issues. Conmiunity-based organizations arc particularly 
vulnerable to systems that do not provide payment until clients complete their programs, and 
public agencies are often concerned about their ability to enter into any subcontract which cannot 
guarantee that payments for staff expenses will be reimbursed. 

Community-based organizations also tend to indicate that the combination of JTPA 
performance measures and performance contracting presses them to serve people with fewer 
barriers to employment than they would choose to serve. For example, a 1984 survey of more 
than 300 community-based organizations conducted for the National Youth Employment 
Coalition found that nearly half (47.9%) of them thought the current system forced creaming. 

Policy lessons (and questions) from performance contracting 

There are some key policy questions which emerge from this picture of performance 
contracting, both from the point of view of JTPA itself and from that of anyone looking at JTPA 
for applicable lessons for another program. Here are a few: 

• Does the approach impel undcsired targeting of clients? 

• Does it affect the quality of services by stressing inexpensive or short-term programs? 

• Does it allow or even foster abuse by lumping together service and administn tive costs? 

Although the term "'creaming" has no universally accepted definition, it is generally used to 
mean avoiding service to clients who are the hardest to place. The riiscussion vis a vis JTPA has 
taken on a new subtlety, not generally present in CETA. The original legislation requires that 
services be offered to "those who can benefit from, and those who arc most in need of, such 
opportunities . . (Sec. 141(a)). Neither term is defined by the Act, but that set of phrases has 
set up the basic dialogue about whom JTPA is to serve. 
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It is clear from every major study undertaken of JTPA that both client selection and 
program design have been influenced by some combination of performance stmdards and 
contracting procedures* with local and state officials clear from the beginning that they believed 
that the law intended them to serve, as one SDA administrator described it, **. . . persons first, 
who are eligible, and secondly, who need what we have to offer.** (Grinkcr- Walker, 1985). 
Anodicr said, in reply to a question about creaming, **we are simply not concerned with the 
issue. Performance-based contracting leaves us no alternative but to avoid the high-risk client" 
(Ibid.) 

Another, interviewed this summer, said, **Contractors are going to get paid on the basis of 
placement, period. You're going to make your money on those easiest to place, just those the 
program ought to be serving. JTPA is a perforaiance-driver program which tries to get people 
through programs as quickly and efficiently as possible and into jobs. If you're going to do that 
you have to look for applicants who are going to succeed" (Brandeis interview, 1988). 

It seems that many in the system do not r jgard the phenomenon as a problenL That was 
true of a slim majority of SDAs in our sample (54.09&), who denied that there was any problem 
of creaming in their own agencies. However, a majority of those who admitted that creaming 
was a problem (55.2%) attributed it to performance contracting, as opposed to only about a 
quarter (23.0%) who attributed it to performance standards, and about two-fifths (39.6%) who 
felt that their testing and assessnnent procedures led to creaming. 

Most of those we surveyed thought that treating the question of one of "creaming or not 
creaming" was too simple. The same majority who thought that creaming was a problem they 
had encountered also thought that JTPA's performance criteria and contracting procedures had 
also had an impact on the design and content, not just the duration of program services. 

On the other hand, as in the case of performance standards, it appears that the impact on 
services need not be negative. Several local officials suggested that performance contracting can 
be used to achieve any desirable end, including service to the more at-risk of the eligible pool of 
potential clients. Not all SDAs arc using performance coniracts simply to maximize placements 
and minimize costs. Some are integrating other performance objectives — such as targeting on 
specific populations or providirig incentives for certain skill areas — into performance contracts 
with good effects. 

Brandeis' Center for Human Resources, the Washington-based Center for Remediation 
Design, and others have been urging SDAs to take a "tiered" ap; roach to program design and 
client selection, which uses performance contracts to produce a i.iix of harder-to-scrvc clients 
being served with more extensive, expensive programs, together with other programs involving 
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more employable clients in shorter-tcrm» less expensive programs. Botli suggest that it is 
possible to tie performance to these other objectives* while still achieving pcnormancc results 
which meet or exceed required SDA averages (Youth programs. Winter, 1987). 

There arc other arguments, to be sure, about performance contracting. The U.S. 
Department of Labor, apparently suspecting that some subcontractors, far from suffering from 
the constraints of rigorous contract requirements, are pocketing profits from quick placement 
programs operated under fixed unit price performance conU',iCts, is currently proposing rules 
which tighten up procedures and impose stringent audit and monitoring requirements. Moreover, 
the suspicion persists that performance contracting has emerged largely as a ruse for SDAs to 
pocket all the allowable administrative costs (15% of total Title n-A funds), while burying costs 
of management at the subcontract level in the "ttaining" category, thereby creating a false 
impression about what it costs to manage JTPA. Both suspicions appear to have some merit, 
though it appears that only a very few local subcontractors are pocketing any serious profits, 
while it is almost certainly true that management and administrative costs are hidden in most 
performance-based subcontracts. 

The policy point, however, remains the same. In the words of one SDA administrator we 
intcr/icwcd, "performance conducting is a tool — nothing more, nothing less. If I'm any good, I 
can use that tool to cream so I can exceed performance standards and bury admin, costs or else I 
can use it to serve special populations in this town that need our help and to hold our contractors 
accountable for their program and fiscal performance. It's value-neutral; it all depends on what 
you use it for" (Brandcis interview, 1988). 

Competency Systems: the search for individual bottom lines 

In an ideal, centrally-based, outcome-directed system where service delivery was 
decentralized, one could imagine that performance measures issued nationally, adjusted and 
enforced at the state and local levels, would be reinforced by contracts which incorporated the 
same objectives with service delivery agencies.Thesc agencies would include in their 
instructional and service plans the building blocks which would produce in clients the individual 
learning and behavior which were required. Indeed, something like that is the vision behind 
JTPA. But only for youth have those outcomes been any more programmatic than placement and 
cost. The competency movement has only in rare instances hit the adult training system. In this 
respect, adult vocational education is considerably more developed than JTPA. 

The 1982 law provided that meeting "PIC-approved competencies" would qualify a youth 
as a successful graduate of JTPA. As the Department of Labor further defined it, this meant that 
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SDAs and states could define as a "positive tcnnination" aU youth who met at least one of the 
local PIC'S criteria of competence in '^pre- employment or woric maturity skills," in "b-wic skills," 
including reading and math, or in "job specific skiUs," such as might be gained in an 
occupational training program or through on-die-job training. 

Due at least in part to the changes in pctfonnancc measures for youth and strong 
encouragement from state level JTPA ofRcials, virtually aU Service DeUvery Areas have adopted 
PIC-approved competency systems for prevocational outcomes, and smaller proportions have 
done so for outcomes that are more closely related to the needs of specific employers. As patient 
advocates for more competency-based programming in employment and training programs, we 
were surprised at the extent to which the concept had penetrated the system, but disappointed that 
it had not taken hold much beyond the pre-vocational outcome* taught in short-term, pre- 
placemcnt programs like job search assistance and pre-employment training. A 1985 survey by 
the National Alliance of Business documented that 81% of SDAs contacted had some form of 
competency-based program in place and another 9% had plans for doing so. As shown in Exhibit 
6, we hoped to go just a step deeper, to find out in what areas competency-based programming 
was taking place. 

Exhibit 6 

Use of competency systems in youth programs 



Brandeis Survey 
(PY 1988) 

NAB Survey 
(PY 1985) 

In place 
In place or 
planned 



Woric 
Maturity 

92.2% 



Pre- 

emp't/ Basic 

Educat 

Skills 

81.2% 



Job Job 

Specific Specific 

Skills Skills 

(Classroom) (OJT) 



31.3% 



54.7% 



81.3% 
89.4% 



Ten (12%) of the SDAs wc spoke to said they "were considering," or ''had plans" to 
implement some form of competency-based programming for adults, but none reported that they 
had done so. Three reported that they "thought" that one or another of their subcontractors— all 
community colleges— operated their classroom training for both youth and adults as competency- 
based, but none were certain. Literally everyone we spoke to who had an opinion on the subject 
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agreed that competency-based programming was "more appropriate" for youth, while harder- 
edged outcome measures like job placement, wages and retention were more suited to their adult 
program graduates. One SDA director reported having proposed to her Private Industry Council 
that they explore competency-based outcome measures for contracts to serve adult illiterates (on 
grounds dial near-term placement was not a feasible outcome goal) and was accused by one 
member of "wimping out on the bottom line." She dropped the proposal 

While there continues to be some talk about adult competencies, ihe development has been 
real only in t^ e youth area. There, for the first time, and after several years of wrangling between 
DOL (which wanted competency measures) and the Office of Management and Budget (which 
resisted them), a new measure, caUed "employability enhancement rate** has been approved for 
measuring youth outcomes. Part of the "positive termination" rate, it means that for the first time 
in Program Year 1988 local programs, in states where the governor has chosen it as one of the 
eight standards from the "menu," will report their accomplishments in terms of progress towards 
employability, as measured by achievement of certain competencies approved by die Private 
Industry Council. If that seems a littie convoluted in the describing, it is likely to be more so in 
implementation. 

"Employability enhancement," while a goal subscribed to by all the practitioners to whom 
we spoke, is in imple^nentation the conccptiial opposite of the impulse for simplicity striven for 
in the initial performance standards. And states are approaching its adoption cautiously. Fewer 
than 50% of the SDAs we surveyed reported plans to use the standards tiiis year. rcflccting the 
number of states which have autiiorized payn-^nt according to tfiat outcome, and only 10% more 
expect to incorporate die standard next program year. Why? "It's complicated," said one SDA 
director, " and my PIC is used to— no, insists on— simple." 

Why is it complicated? Basically, it appears that SDA staffs, accustomed to the more 
straightforward measures of performance like placement and wage levels, are daunted by the 
planning tasks involved in development of genuine competency systems. This factor appears to 
account for the use of competencies primarily in short-term programs to date, as well as the 
caution exhibitca in going further. Designing a competency system in JTPA will require several 
steps, at least the following: 

• surveying local employerneeds 

• designing a system of program services which appear likely to respond to those needs 

• developing a "map" of the competencies which will have to be provided to participants 
through experience and instruction 

• developing adequate assessment, monitoring, and certification mechanisms 
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• developing mechanisms and selecting instruments for client appraisal and for progress- 
and post-testing ot client acconq)lishnients 

• developing or acquiring cuiriculum that matches employer needs to client status and 
abilities 

• developing contracting mechanisms which reach past placement and cost measures and 
include documentation of client or student progress. 

Perhaps more important, because the technicalities of such a task are certainly not beyond 
present capacities— even though they are hard— SDAs will need to convince PICs. 
subcontractors, and not least themselves, that it is worth forgoing short-term, quick client 
placement for longer, probably more expensive training programs which may not be as readily 
bottom-line-measurable as they have become accustomed to. 

They will need help and political support even if they develop the local will and apply 
themselves to the technical task. TTicre is some promise of help. Apparently DOL is looking to 
this approach to temper the problems it perceives with the performance standards and the impact 
of performance contracting, and plans to target technical assistance resourees to the task. Most 
states appear to have offered both encouragement and help for SDAs who want to add to their 
present pre-cmployment/work maturity systems. Roughly four of every five SDAs in our sample 
(79.7%) reported receiving encouragement from their states to develop and implement systems to 
assess achievement of specified competencies. Moreover, the encouragement has been an 
ongoing process, having been felt for three or more years in two-thirds (67.4%) of the SDAs 
which reported such encouragement Not surprisingly, state efforts to promote competencies 
tended to focus on service to youth, but there are some instances of broader encouragement, 
e.g., to explore competencies for all client groups. 

The basic policy question for JTPA regarding competency systems is a variant of the ones 
raised by performance standards and contracting. That is, will the system accept the 
accomplishment of measurable competencies by participants as legitimate objectives, and 
therefore acceptable outcome measures? Or must the process be kept as simple and measurable as 
a few placement and cost measures imply? So far, the answer from federal policy makers has 
been timid. Yes, they say, competencies are an acceptable outcome measure for youth, but not 
for adults. But even for youth, it may only be pan of this "positive termination" measure, and it 
doesn't really "count" as much as entered employment rates. The fear seems to be that if 
competencies are an accepted outcome, local SDAs will compromise the harder-line measures of 
cost and placement rates, and substitute competencies in too many cases. 
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Conclusion and Recommendations 

The JTPA drive for performance appears to be working pretty well. The performance 
measures, while possibly too few and too simple, arc the right ones, and recent federal changes 
appear to compensate fc some of the shortcomings of the earb'er simpUcity. The approaches to 
state and local adjustment are vciy important local tailoring provisions, but they appear not to be 
working very well, and much more thought and technical assistance will be required to make 
them effective. Some of the costs and quality shortfalls which may be concealed by performance 
contracting are worrisome, but the basic approach makes sense and is unlikely to be thrown out 
even though it is currently under sharp scrutiny. The capacity to turn national and state outcome 
measures and standards into specifications for local programs, and to incorporate them into 
contiacmal arrangements is a signal advance, and ought to be maintained. The potential for abuse 
is clearly there, however, and without new rules, incentives, and technical assistance, local 
contracting procedures could, paradoxically, work in the long run against achievement of a 
genuinely outcome-driven system for employment and training for the disadvantaged. 

The most difficult, and in my view progranmiatically significant development, competency- 
based training and education, has spread wide but not deep in the system, reflecting the extent to 
which JTPA is still smarting from its CETA legacy. Federal and local officials alike arc 
determined to maintain what feels like a hard-won reputation for tough-minded measures and 
standards. Accordingly, any development which may complicate the bottom-line picture for 
which they strive is viewed with mistrust Nonetheless, there is some discomfort that the 
performance measures und standards, contracting, and budget reductions may have conspired to 
tilt the system towards the most job-ready and away from the more complicated clients (who will 
take longer, cost more, succeed in smaller numbers . . .). Whether it is true is haid to prove. 
Logically, it is likely that the tilt has taken place, although it is splitting a pretty fine hair in 
general society to talk about "creaming" from members of the pool of people whose economic 
status makes them eligible for JTPA services. 

What are the implications for vocational education for adults? Pint, JTPA has demonstrated 
that it is possible to establish a national program substantially driven by clearly-specified, 
measurable outcomes. Moreover, it is possible to tic those outcomes to funding, not only at the 
federal and state levels, but even down to the level of actual service delivery through 
performance- based contracts with educational institutions or community agencies. As policies 
which will underlie vocational education are developed, JTPA's experience should encourage 
planners that a focus on outcomes can be achieved 
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Secondly, the performance objectives and the tools -flesigned to achieve them are never truly 
"value-neutral,'* as one of our SDA interview Subjects asserted, but they are relatively 
manipulable. For example, while JTPA observers arguef'^ver creaming from among an eligible 
population of America's disadvantaged, vocational educ^on has not ever really decided to what 
extent it ought to target certain parts of the populati«fe The lesson from JTPA is that both 
national and state standards and local contrr.?r<ng pioMbres can be designed and managed to 
accomplish targeting objectives. ""i 

Third, while performance contracting has some proWems, JTPA's experience with them is 
encouraging. It appears that it is possible to write local contracts with educational institutions and 
community agencies which link payment to speciHc outcome objectives, and to be planful in 
pursuing a mixed strategy in a community that caM produce a range of outcomes, tailored to £nd 
produced by different agencies according to their capabilities and connections with the 
community. Performance contracting may be the mechanfem by which state officials can, if they 
wish, spread training responsibility among, say, cdiflmunity colleges, community-based 
organizations, proprietary schools and even businesses, with public attention to their relative 
achievement of performance objectives. CarefuUy handled, ocrformance contracts can even 
change the nature of the local turf dialogue among training providers. 

Fourth, it is probable that vocational education Yds more to offer JTPA in the area of 
competency-based curriculum and program design than the reverse. JTPA is working hard in the 
youth area, and the new performance measures being pi«posed for future years may open up the 
subject for adults as well. But the technology is little detcloped. Do vocational educators know 
more? I susprct so. We ought to look there for lessons, as welL 

Finally, it may be possible, if vocational educatioif continues to move in the direction of 
outcome-based management, for the two systems to become more integrated with each other. 
Those of us who work in one part of the field or the other may understand the distinctions, but 
the general public docs not, and certainly Congressional and state legislators who ultimately 
make the policies and vote on whether and to what extent to fund them are not comfortable that 
one hand knows what the other is doing. Moving both systems in the direction of outcome- based 
programming and management may bring the two closer together to the benefit of both. 
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